Skip to content

tools(dnn-i18n): #457 content-type extractor + reimport dry-run (fixture, Option C tooling) - #524

Merged
jsboige merged 1 commit into
masterfrom
tools/457-dnn-strings-extract-reimport
Jun 17, 2026
Merged

tools(dnn-i18n): #457 content-type extractor + reimport dry-run (fixture, Option C tooling)#524
jsboige merged 1 commit into
masterfrom
tools/457-dnn-strings-extract-reimport

Conversation

@jsboige

@jsboige jsboige commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

What

#457 DNN i18n tooling — bricks 1 + 3 (content-type extractor + re-import dry-run verifier) + round-trip DoD test. Dispatched by ai-01 (msg-…141408, non-gated #457 tooling while #131 exec waits on jsboige's target decision).

Key finding that shaped the work

Brick 2 (DatasetUpdater Option C config) was ALREADY merged in #487 (ca9a8640) — entity DnnUiString, KnownDataSets.DnnUiStrings, prompts, task config (DatasetUpdaterRootConfig.cs:2636-2696, Enabled=false, gpt-5.5). The investigation (SDDD triple-grounding) confirmed the config-only gpt-5.5 path is canonical. So this PR adds the missing tooling around it rather than rebuilding the config:

Brick What Status
(2) config Option C gpt-5.5 task ui./res. ✅ already done (#487)
(1) extractor content-type → CSV (codifies PHASE1 manual audit) 🆕 net-new
(3) re-import verifier key-set diff + payload render (dry-run) 🆕 net-new

Brick 1 — extract_dnn_ui_strings.py

Parses the Argumentum 2sxc .cshtml templates, emits the dnn-ui-strings.csv dialect (key,context,source_file,fr,en,ru,pt,es,ar,fa,zh,notes):

  • ui.* = anchor-based (NOT free-text): each entry declares a verbatim anchor that must exist in the named source. Fails loud (exit 2) if a refactor removes a string — never silently drops a row, never fabricates.
  • res.* = @?Resources.<Key> regex capturing both @Resources.X (directive) and @Html.Raw(Resources.X) (nested expression). The nested form recovers RuleMemoInstructions (the one a naïve @Resources. regex misses). fr left empty + flagged DB-only (canonical values live in SQL/2sxc App Resources — pattern appris).

Brick 3 — reimport_dnn_ui_strings.py

  • verify — key-set diff vs reference. KEY-SET match is HARD (exit 1 on divergence). res.* fr empty-in-extract is by-design WARNING.
  • reimport — renders the canonical DNN/2sxc payload to stdout only (never writes — live apply is DB/RDP-gated, jsboige).

DoD proof — test_roundtrip.py

[1/4] extract OK: 10 rows (2 ui.*, 8 res.*)
[2/4] key set OK (10 keys)
[3/4] verify OK (key sets match, res.* fr empty by design)
[4/4] reimport dry-run OK (payload rendered, nothing written)
ROUND-TRIP DoD PASS — exit 0, zero prod mutation

Anti-fabrication (proven, not asserted)

  • Negative test: breaking a ui.* anchor → exit 2 (verified, not piped).
  • Cross-validated vs REAL prod templates: extractor run on DNNPlatform/Portals/1/2sxc/Argumentum/ yields 10/10 keys matching docs/dnn-localization/dnn-ui-strings.csv. Only delta = res.* fr (empty-in-extract by design — DB-only).

Gate boundaries (HARD)

🤖 Generated with Claude Code

…ure, Option C tooling)

Bricks 1 + 3 of the #457 dispatch (ai-01 msg-…141408). Brick 2 (DatasetUpdater
Option C config, gpt-5.5 ui.*/res.*) was ALREADY merged in #487 (Enabled=false) —
not rebuilt; this PR adds the missing tooling AROUND it.

Brick 1 — extract_dnn_ui_strings.py: reusable content-type -> CSV extractor that
codifies the manual PHASE1 audit. ui.* = anchor-based (fails loud if a hardcoded
string disappears, never invents rows); res.* = @Resources.* references with DB-only
fr flagged honestly. Regex @?Resources. captures both directive (@Resources.X) and
nested-expression (@Html.Raw(Resources.X)) forms — recovers RuleMemoInstructions.

Brick 3 — reimport_dnn_ui_strings.py: `verify` (key-set HARD diff vs reference) +
`reimport` (render canonical payload to stdout, NEVER writes). res.* fr empty-in-extract
by design (DB-only values); KEY-SET match is the hard contract.

DoD: test_roundtrip.py proves extract -> verify -> reimport on a committed fixture,
zero prod mutation (PASS, exit 0). Cross-validated vs REAL prod templates: 10/10 keys
match docs/dnn-localization/dnn-ui-strings.csv.

Anti-fab: anchor-based ui.* (negative test proven exit 2), no free-text fabrication,
honest DB-only flagging. Gate-safe: pure tooling, no prod/DB/CSV/config mutation, does
NOT touch dnn-ui-strings.csv (po-2024 lane #490). Stdlib Python only.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@clusterManager-Myia

Copy link
Copy Markdown
Collaborator

[NanoClaw]

Clean i18n tooling (bricks 1+3) for DNN localization: anchor-based ui.* extractor (anti-fab: fails loud if anchor disappears, never invents rows), res.* reference scanner (DB-only values honestly flagged), CSV reimport dry-run verifier, and a round-trip test with fixture. Stlib-only Python, cross-validated against prod templates (10/10 key match). Gate boundaries explicit: no DB touch, no live reimport, no CSV modification, no DatasetUpdater config change. Subprocess only used to run its own test script. Good README with anti-fabrication guarantees. LGTM.

@jsboige
jsboige merged commit bf5d733 into master Jun 17, 2026
3 checks passed
@jsboige
jsboige deleted the tools/457-dnn-strings-extract-reimport branch June 17, 2026 17:08
jsboige added a commit that referenced this pull request Jun 17, 2026
…tractor) (#525)

Closes the "inventory content-types" idle task (ai-01 msg-…141408). Extends
PHASE1-content-audit.md (#461) with a complete localization scope map and feeds
the extractor (PR #524, merged).

5 content-types mapped (Argumentum-app scope only; DNN framework out-of-scope):
- A custom-template hardcoded UI strings (2 keys) — extractor-done
- B @Resources.* keys (8, values DB-only) — extractor-done (keys + DB-only flag)
- C 2 static HTML content pages (fallacies.html ~24K chars + MariagePourTous ~13K,
  standalone FR prose, NOT CSV-driven) — repo-extractable gap, recommend separate
  document-tier translation lane (not the ui.*/res.* string task)
- D 2sxc content items (glossary/FAQ/homepage/App resource VALUES) — DB-only, the bulk
- E DNN page settings — DB-only. Both D+E need portal export (jsboige, gated).

Findings: 26 2sxc apps installed but only Argumentum is custom (25 stock). The live
« Materiel » coquille is NOT in any repo template (negative grep across Argumentum scope)
→ it's DB-only (content-type D), consistent with the "canonical FR lives in SQL" pattern.

No prod mutation, no DB/RDP, no dnn-ui-strings.csv touch. Pure scope-map doc.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jun 20, 2026
docs(dnn): #457 document-tier translation workflow (Phase 2-4 design)

The Phase 1 inventory (457-site-content-type-inventory.md) named WHAT to
translate but deferred the HOW for content-type C (the 2 standalone FR
HTML pages, ~37k chars of prose). The string-tier toolchain
(tools/dnn_i18n/, PRs #524/#487) handles ui.*/res.* keys but does not
fit prose. This doc designs the document-tier lane: extraction ->
DatasetUpdater -> re-import.

Scope: DNNPlatform/fallacies/fallacies.html (81 KB, structured charte)
+ MariagePourTous.html (38 KB, prose).

Design:
- Phase 2 CSV schema: segment-oriented (segment_id = file#element-index),
  source_path + selector for deterministic re-import, inner-text fields
  per language. NOT the key->value string-tier dialect.
- Phase 2 DatasetUpdater task shape: document-tier variant vs string-tier
  (prose prompt, char-budget chunking, gpt-5.5, reasoning effort low,
  max_output_tokens >=7000, Enabled=false until GO).
- Phase 1.5 extractor: the net-new gating tool (html.parser stdlib,
  file-specific selectors h2/h3/li vs p/h2/h3, inner-text extract +
  selector store for re-wrap). Nothing translates until this exists.
- Phase 4 re-import: selector-based patch, dry-run verifier (same DoD as
  string-tier round-trip), localized HTML files per language.
- RTL/CJK (ar/fa/zh): reuse PR #359 font stacks at page level
  (dir=rtl/lang), template variant decision for jsboige.
- Sequencing: extractor is critical path.
- 4 open questions for jsboige (inner-text vs outer-HTML, one-template vs
  per-lang, are pages still served, GO to build extractor).

Index README #554 updated to reference the new doc in supporting-docs
table. Pure design/references -- no prod mutation, no extraction
executed, no DatasetUpdater enabled.


@

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants